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Abstract 

We investigate the relation between frequentist and Bayesian approaches. Namely, 

we find the "frequentist" Bayes prior 7r/(A, x ft s ) = °J^ x - — (here f(x,X) is 

the probability density) for which the results of frequentist and Bayes approaches 
to the determination of confidence intervals coincide. In many cases (but not al- 
ways) the "frequentist" prior which reproduces frequentist results coincides with the 
Jeffreys prior. 
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One of the standard problems in statistics |T] is an estimation of the values of unknown 
parameters in the probability density. There are two methods to solve this problem - the 
frequentist and the Bayesian. 

In this paper we investigate the relation between frequentist and Bayesian approaches. 

rx ob3 df(x,\) dx 

Namely, we find the "frequentist" Bayes prior 7Tf(X,x obs ) = — ~ x tl ■ a \, — (here f(x,X) 
is the probability density) for which the results of frequentist and Bayes approaches to 
the determination of confidence intervals coincide. In many cases (but not always) the 
"frequentist" prior coincides with the Jeffreys prior. Note that in ref.[2] the relation 
between frequentist confidence intervals and Bayesian credible intervals has been found 
for probabilities densities of the special type f(x, A) = &(x — A) and f(x, A) = jF(j )■ 

As an example consider the case of random continuous observable — oo < x < oo with 
the probability density f(x, A)c. 

In Bayesian method [TJ [3] due to Bayes theorem 

mm= zsmA (1) 

the probability density for unknown parameter A is determined as 

P(M x obs) — f +oo 77 T7T 777TT77 ■ (2) 

J-oo fixobs, A )tt(A )dX 

Here x Q b s is the observed value of the random variable x and 7r(A) is the prior function. 
In general the prior function vr(A) is not known that is the main problem of the Bayesian 
approach. Formula (2) reduces the statistics problem to the probability problem. The 
probability that parameter A lies in the interval Xdown < A < X up is |f 



do u> n 



P(^down < A < X up ) = I p(X\x obs )dX = 1 - a(X up , X down \x obs ) , (3) 

where 



c( A^p, Xdown | ^obs) ( A U p | X obs ) -\- r ) / ( Xdown \ •Eobs) j (4) 

P(K P \xobs) = /. p(X\x obs )dx, (5) 



-'^Here oo < A < — oo is some unknown parameter and f(x, X)dx = 1. 

2 Usually a{\ up ,\down\xobs) is taken nondependent on X up , Xdown and equal to 0.05. 



l(X down \x obs ) = / p(X\x obs )dx = 1 - I3(\ down \x ohs ) , (6) 

J — OO 

The solution of the equation (3) is not unique. The most popular are the following 
options p]: 

1- ^down = — oo - upper limit. 
2. X up = oo - lower limit. 



3. Jl^ u '" p(X\x bs)d\ = J^ p p(X\x b s )d\ = f - symmetric interval. 

4. The shortest interval - p(X\x b s ) inside the interval is bigger or equal to p(X\x b s ) 
outside the interval. 

In frequentist approach the Neyman belt construction jl](see Fig. 1) is used for the 
determination of the confidence intervals. 
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Figure 1: Neyman belt construction 



Namely, we require that 



•^down (A) 



f(x, X)dx = 1 — a . 



(7) 



or 

(A) 



/ f(x,X)dx + f{x,X)dx = a. (8) 



It should be stressed that in general case both (X) f(x, X)dx and J^ wn ^ f( x , \)dx can 



/ /(x, A Mp )ofe = 1 - a , (9) 



'Xup(X) 

^Xup(^) 

depend on A but their sum does not depend on A (see eq.(8)). 

The Neyman equations for the determination of upper and lower limits X up and Xd own 
on parameter A have the form 

C x up{X U p) 

x obs 
f x obs 

I f{x, X down )dx = 1 - a (10) 

x down {^down) 

and they determine the confidence interval of possible values 

Xdown < A < X up (11) 

of the parameter A at the (1 — a) confidence level. 

In this paper we shall consider the case when J^ p ^ X ) f( x , X)dx does not depend on 
A. For instance, the options x up (X) = oo, Xd OW n(X) = — oo and j™ p ^f{x,X)dx = 
f*£ > wn ^ f(x, X)dx = | correspond to the cases of upper limit on A, lower limit on A 
and symmetric interval correspondingly. As a consequence of the eqs.(8-ll) and our as- 
sumption on nondependence of r\) f(x, X)dx on A we find that 

/x obs roo 
f(x,X up )dx+ / f(x,X doW n)dx . (12) 

To find the relation beetween frequentist and Bayesian approaches we have to find the 
prior for which the formulae (4-6) and (12) coincide. Namely, we require that 

/^down roo r%obs roo 

p(X\x obs )dX+ p(X\x obs )dX= f(x,X up )dx+ f(x, X down )dx . (13) 
- OO J X up J —CO ^ x obs 

The solution of eq.(13) is 

p(X\x obs ) = - f_2 '^p-dx . (14) 

Formulae (13,14) demonstrate the equivalence of the frequentist approach and the Bayes 
approach with the prior function 

rx ob3 <9/Q,A) r 
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Note that in the limit Xdown — > — oo and X up — > oo full probability must be equal to 
one, namely 

lim / [f(x,X down ) - f(x,X up )]dx = 1 . (16) 

^dnwn.— * — 00,A^p— >OQ J — OO 



As a consequence of the equation (16) we find that 



Km / f(x,X down )dx = 1, (17) 

s^ri.nnm ^ OO , 



OO 



lim / f(x,X up )dx = 0. (18) 

A^p — ^oo J — oo 

Consider several examples. For the probability density 

/(x,A) = $(x-A) (19) 
as a consequence of the formulae (14-15) we find that 

Pj(X, x obs ) = §(x obs - A) , (20) 

7T/(A, Xobs) = 1 . (21) 



Note that for the distribution (19) the Jeffreys prior [5J tt(A) ~ y'f/f^ /(ar, A)( 9fa( gf' A)) ) 2 cfa] 
const does not depend on A, i.e. for the distribution (19) the frequentist approach is equiv- 
alent to the Bayes approach with the Jeffreys prior vr(A) = const [2]. 

Consider the case with the parameter A > b. Here b is some fixed number □. Such 
situation arises when we measure signal s in the presence of nonzero background b and 
X = b + s, b > 0, s > 0. The parameter A lies in the interval b < X < oo. The direct use 
of the formulae (12,13) leads to the inconsistency. Namely, we find that the probability 

roo 

P(b < A < oo) = J $>{x obs - X)dX < 1 (22) 

that contradicts to the postulate that the full probability must be equal to 1. At the 
frequentist language the inequality (22) is the consequence of the fact that 



roc 

/ f(x, X down = b)dx 7^ . (23) 

J x obs 



3 In ref. [B] normal distribution N(x,^i,a 2 ) with additional constraint \x > has been studied. 
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To obtain the correct solution we must use the language of the conditional probabilities. 
Really, the probability that parameter A lies in the interval Ao < A < Ao + dX is equal to 
&{xobs — Xo)dX. The probability that parameter A lies in the interval Ao < A < Ao + dX 
provided A > b is determined by the formula of the conditional probability 

P (A < A < A„ + d A|A > t) = P(A ° |^ + fU) = " X 'mx ■ < 24 > 

P(X > b) J b ®{x obs - X)dX 

So we see that condition A > b leads to the appearance of additional factor $(x &s — 

X)dX in the denominator of the formula (24). This factor restores the requirement that 

full probability P(b < X < oo) = 1. For instance, the probability that signal s is less 

than So is determined by the formula 

pi* < *) = '\! { :r~ b 7^ s < 25 > 

J b $(x obs - AjdA 

and it coincides with the corresponding formula of the CL S method [3, [8]. 
For the probability density 

F(x,A) = i$(^) (26) 

we find that 

Pf{\x obs ) = X -f®{ X -f), (27) 

n f (X,x obs ) = ^f. (28) 

Again in this case the prior (28) coincides with the Jeffreys prior [2]. 
Consider the probability density 

f{x,\) = 0(x)P{[x],\), (29) 

where 

P([x],A) = pe- A (30) 

is the Poisson distribution and [x] is an integer part of x (for instance, [2.33] = 2). Using 
formulae (14,15) one can find that for the probability density (29) and x obs > 

Pf(x bs, A) = P([x obs ] - 1, A) + {x obs - [x obs ])(-P([x obs ] - 1, A) + P([x obs ],X)) , (31) 



4 For the probability density (29) f(x, \)dx = YZZ™ P ( n > A ) = L 
5 For x obs < 1 Pf(x obs ,X) = P(0,A) and 7T/(A, x b s ) = 1. 



n f (X,x ohs ) = 1^1 + (x obs - [x o6s ])(-I^l + 1) . (32) 

The Jeffreys prior for the distribution function (29) coincides with the Jeffreys prior for 
Poisson distribution and it is proportional to 7r(A) ~ So we see that for the probability 
density (29) the "frequentist prior" (32) and the Jeffreys prior are different. 

Consider now the case when ^ f(x, \)dx depends on A. Instead of x up (X) and 
XdownW we can find another x up (X) and x down (X such that x up (X up ) = x up (X up ) = x obs , 

%down\^down) ^downi^down) %obs &Ild 



/ f(x,\)dx= f(x,X up )dx, (33) 

J -co J CO 

rco rco 

f(x,\)dx= f(x, \ down )dx. (34) 

Jx, (A) Jx nh „ 



/ x 'upW , 
f(x, \)dx = 1 — a , (35) 



X down^ J' x obs 

For the confidence intervals defined by new trajectories x' up (X) and x down (X) the Neyman 
equations have the form 

X down 

/^obs fCO 
f(x,X up )dx + / f(x, \ d own)dx . (36) 
-OO J Xobs 

and the equations (14,15) are valid. So we see that for the Neyman belt construction with 
x' (A), 2^ oum (A) and the integral (x) f(x, X)dx nondependent on A the frequentist 

" down^ ' 

approach with a given by the exppression (36) and the Bayesian approach with prior 
function (14) coincide. 

In conclusion let us formulate our main result. For the particular case when ,^ f(x, \)dx 

does not depend on A we have found the "frequentist" Bayes prior 7T/(A, x obs ) = °°^ x - ^ — 

for which the results of the frequentist and the Bayes approaches to the determination of 
confidence intervals coincide. In many cases (but not always) the prior which reproduces 
frequentist results coincides with the Jeffreys prior. 

This work has been supported by RFBR grant N 10-02-00468. 
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